Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Move to Polkadot SDK #461

Open
wants to merge 16 commits into
base: develop
Choose a base branch
from
Open

Move to Polkadot SDK #461

wants to merge 16 commits into from

Conversation

kayabaNerve
Copy link
Member

Running cargo +nightly clippy --all-features --all-targets on remove-subxt (which this builds off) compiles 1215 crates. This compiles 1195 and includes a lot of version increases.

This prematurely merges paritytech/polkadot-sdk#1631 and paritytech/polkadot-sdk#1313 into an experimental branch of serai-dex/polkadot-sdk. The former is needed, as cargo can't resolve the crate graph if we pull in the old libp2p with its older crypto dependencies. The latter is solely a nice to have.

This PR exists to evaluate if the CI passes or if further edits are needed.

@kayabaNerve kayabaNerve added improvement This could be better node labels Nov 28, 2023
@kayabaNerve kayabaNerve force-pushed the polkadot-sdk branch 2 times, most recently from b64ef5d to b06e8a8 Compare November 28, 2023 07:34
@kayabaNerve
Copy link
Member Author

2023-11-28 08:28:12 Error running offchain workers at 0x3514ed54f2dbfa21208dfa1799a87e3b9aa5f2d1e7d4f5055a3e60d59b6914d9: Execution failed: Other error happened while constructing the runtime: failed to instantiate a new WASM module instance: maximum concurrent memory limit of 1 reached for stripe 0

Seems to be a wasmtime misconfig somewhere.

====================

Version: 0.1.0-unknown

   0: sp_panic_handler::set::{{closure}}
   1: <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call
             at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/alloc/src/boxed.rs:2021:9
      std::panicking::rust_panic_with_hook
             at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/std/src/panicking.rs:735:13
   2: std::panicking::begin_panic_handler::{{closure}}
             at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/std/src/panicking.rs:601:13
   3: std::sys_common::backtrace::__rust_end_short_backtrace
             at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/std/src/sys_common/backtrace.rs:170:18
   4: rust_begin_unwind
             at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/std/src/panicking.rs:597:5
   5: core::panicking::panic_fmt
             at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/core/src/panicking.rs:72:14
   6: core::panicking::panic
             at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/core/src/panicking.rs:127:5
   7: <sc_service::client::client::Client<B,E,Block,RA> as sc_client_api::backend::LockImportRun<Block,B>>::lock_import_and_run::{{closure}}
   8: sc_consensus_grandpa::environment::finalize_block
   9: <sc_consensus_grandpa::environment::Environment<B,Block,C,N,S,SC,VR> as finality_grandpa::voter::Environment<<Block as sp_runtime::traits::Block>::Hash,<<Block as sp_runtime::traits::Block>::Header as sp_runtime::traits::Header>::Number>>::finalize_block
  10: <finality_grandpa::voter::Voter<H,N,E,GlobalIn,GlobalOut> as core::future::future::Future>::poll
  11: <finality_grandpa::voter::Voter<H,N,E,GlobalIn,GlobalOut> as core::future::future::Future>::poll
  12: <sc_consensus_grandpa::VoterWork<B,Block,C,N,S,SC,VR> as core::future::future::Future>::poll
  13: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll
  14: <futures_util::future::select::Select<A,B> as core::future::future::Future>::poll
  15: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll
  16: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll
  17: <sc_service::task_manager::prometheus_future::PrometheusFuture<T> as core::future::future::Future>::poll
  18: <futures_util::future::select::Select<A,B> as core::future::future::Future>::poll
  19: <tracing_futures::Instrumented<T> as core::future::future::Future>::poll
  20: tokio::runtime::park::CachedParkThread::block_on
  21: tokio::runtime::context::runtime::enter_runtime
  22: <tokio::runtime::blocking::task::BlockingTask<T> as core::future::future::Future>::poll
  23: tokio::runtime::task::core::Core<T,S>::poll
  24: tokio::runtime::task::harness::Harness<T,S>::poll
  25: std::sys_common::backtrace::__rust_begin_short_backtrace
  26: core::ops::function::FnOnce::call_once{{vtable.shim}}
  27: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
             at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/alloc/src/boxed.rs:2007:9
      <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
             at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/alloc/src/boxed.rs:2007:9
      std::sys::unix::thread::Thread::new::thread_start
             at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/std/src/sys/unix/thread.rs:108:17
  28: <unknown>
  29: clone


Thread 'tokio-runtime-worker' panicked at 'attempt to divide by zero', /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/core/src/ops/arith.rs:484

@kayabaNerve
Copy link
Member Author

kayabaNerve commented Dec 4, 2023 •
edited
Loading

This force push doesn't fix any of the prior observed issues and is just misc improvements. It's most notable for updating to polkadot-sdk/experimental rebased on top of the current polkadot-sdk/serai and effectively removing directories-next/option-ext via #467.

This brings the entire Serai tree to 1193 crates when running cargo +nightly clippy --all-features --all-targets (develop is 1215).

@kayabaNerve
Copy link
Member Author

Now the issue is InvalidTransaction. Some minor distinction with TX encoding, I guess? Or I updated the runtime but not serai-client?

@kayabaNerve kayabaNerve force-pushed the polkadot-sdk branch 3 times, most recently from 760e5f7 to 70cce9e Compare December 17, 2023 05:16
@kayabaNerve
Move to polkadot-sdk
d038aa9 
Still a WIP, and using an experimental branch with 2 not-yet-merged PRs merged.
@kayabaNerve
Add empty crates to clean up the Cargo.lock
b8e5fa7 
Also removes a now unused RUSTSEC allow in deny.
@kayabaNerve
Set the justification generation period to 1
e78fd12
@kayabaNerve
Correct rebase re: Coordinator Cargo.toml
a5065e5
@kayabaNerve
Add patches for directories/directories-next/option-ext
1820e91 
The rational is detailed in the root Cargo.toml.

While I don't personally mind MPL dependencies, even if I don't prefer them
(they're allowed in the deny.toml for a reason), I do mind the pointless scope
creep and wish to highlight how little it actually used from the crate by
re-defining it as the single function.

We could also fork directories-next, or directories, and remove the usage of
option-ext per dirs-dev/dirs-sys-rs#24, yet that'd be
a much larger task than what was done here.

In the future, it may be beneficial to submit a PR to wasmtime replacing
directories-next with home, a cargo-team maintained library to get the home
directory and associated folders. An example migration can be found at
harryfei/which-rs#80.
@kayabaNerve
Correct justification_generation_period from 0 to 1
f03ca53 
Avoids a divide by zero in sc-consensus-grandpa.
@kayabaNerve
Remove TX_VERSION from substrate/client, as CheckTxVersion was remove…
08adb3a 
…d from the runtime

serai-dex/polkadot-sdk@b892b01 for reasoning.
@kayabaNerve
Patch is-terminal to the std-included IsTerminal
0983a49
@kayabaNerve
Rebase to the latest develop
d8c7465
@kayabaNerve
cargo update to polkadot-sdk/experimental rebased over polkadot-sdk 1…
ec0d38a 
….5.0
@kayabaNerve
Include rust-src, remove simple-mermaid from tree
d62f311
@kayabaNerve
Combine -c flags in lint workflow
5e42fd2
@kayabaNerve
Patch matches, mach
ffae13b
@kayabaNerve
Split the components back up
d149d8a
@kayabaNerve
Correct addition of rust-src component in Dockerfiles
bbad773
@kayabaNerve
Remove sp-crypto-ec-utils patch reintroduced in latest rebase
35265c6
@kayabaNerve
Copy link
Member Author

Due to paritytech/polkadot-sdk#2944, we should probably target 1.10 which predates its merge. That conflicts with how libp2p 0.52 still hasn't been merged into polkadot-sdk (so it'll inherently be a later commit).

We could patch libp2p 0.51 to use dalek 4.0 internally? That'd be the other valid solution.

@kayabaNerve
Copy link
Member Author

Polkadot 1.14 has all the dependency updates we want. 1.15 fixes some bugs I have tracked.

As I said above. litep2p is now in tree and both more performant for Polkadot and unstable (yet present and causing various effects, such as re-arranged internals, accordingly). Each new update will presumably make the internals closer to where they should be and make litep2p more and more stable.

I think, given our timeline and the timeline of releases, 1.15/1.16 would be a fair target.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
improvement This could be better node
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@kayabaNerve